* * ^ 



DOCDMENT SESDHE 



ED lift «I31 

AUTHOR 

TIITLE 

PUB OAT-F"-' 



Erns ppicE 

DFSCPIPTOBS 



r. 



IDENTIFIERS 



A 



. . ' TIJ OOa 997 

Olien, G. N.; And Ot'ft'ers ; 
A Social" Sjrsteas -Approach to Evaluation Research. 
[Apr 75] ' •. 

Tip.;' Hot ayaila-fele in hard copy due to ma-rginal 
legibility of origi|nat . document | ' - 

HF-$0.76'Plus! Postage.^HC Not Available from EDR^.w 
A^ult Educd-tlon Progl\amsi{ jiudienbes; Deliveiry 
Systems; ^sau^ational! Television; Elementary 
Education ;^J^Evalufation; *Inf ormatioTi Di-ssemlmatfon ; 
Kass- FTedial; •'♦^Models;^ Nutrlition Instf\iction; 
6rganizatipn;rPredictiojn;;i ^Program Evaluation; 
Research; Soclial Action; Sacial Systems;* Student ^ 
P^actipi^;. 4^SyBtems Analysis , ^ ^ ... 

Mull]i-gan Btewl ^ ^ ,<» 1 ' 

ostpact; ^ ^ . V^t' * . ' ^ / 

An ^yjJEdrmationVcontrol. systems model for evaluation 
of adult' educfiwtioh -programs is rof fere d and*, illustpa+ed • The model is 
'barsed upon identifying principal subsystems^ ..^uch"as; source, channel 
and audiencer which are involv«p in initiktiolfi, production, delivery 
and- receptip^ :of» e4ucatioi\al ffieksages • Thes^ *su,fasystems are seen as 
separate bui ift^erd^pendent, Having the pow^r sinjgi?y or jointly to" 
make crucial decisions abou4: gfrie ration, dissemination of' -withholding 
of information. The extent to^whlich these subsystems may in fact i 
exert such controls has .been d^^QTistrated in a variety of research 
studi^es, many in the mass communilcati on area. Specific , application of 
Ithe imodel is ill\istrated in th^ eWaluation df the "Mulligan Stew" 
t*ere^ision seri-ea. which, -althoiighldirebted** at *a Wbung audience,, hSs 
implications for evaluat*iqn o{f ^?inyi ^^d'ult educiti<S)n progralf which 
'involves extenarive use of the 'taassiirommunicatibn^ media.^ The study 
included b'efore-after st-udent resj^d^nses in eigit schools gnd 
estimates of such response froW nutrition- specialists,, information 
specialists^ televi;fei6n station^ personnel, and teabhers. In terms, of 
final* otttcora^ the" Bn3.1igan Ste\w results generally\and strongly, / 
support th^e informatioln ccmtrdl'.stst^ms 'model, and\testify to it^^ 
utility In evaluational researcH. ^(R6) ' ' 



ERIC 



* Documents acquired by ERIC include many informal unpublished ^\ • * 
t materials not available firom pther^ sources-^ EBIC ma)ce^. every effort * 

* to obtain the ,best copy availablke* ^Hi^vertheless^, items\of marginal * 
^ reproducibility are often encountered and this affects ^the quality *^ 

* of th.e microfiche and hardcopy reproductions ERIG maKes .available ^ * 

* via tfie ERIC Document Re^rroduction Service ()?DRS)* EDRS \is not * 

* responsible for t:^e quality of the original document. Peprodu^ions * 

igina-1^ * 



sponsible tor t^e quality ot the orig 
* supplied by EDPS are the best that can be made from ,the 



V- 



^Sf&tfkWJMEHXQfi HEALTH. 

-^'lij^lONAL INSTITUTE Of 
* ^€OUCAI^ON 

THiS cWj^fnt ma$ been <jepro 

..Ol>C«i3V^J«ACny AS RECEIVED s^ROM %^ 
▼neT^SCHO)? OflOANIZATlONOR'^lN 
AXiNG >T>.7TV7 5Jq/ V»'EA or OPiNtONS 

l*>tNTOfriC»At >fAT»OVAt »N\T»TOTf 0» 
f Dc<'ATlON S»OSiTi<>S OR PC. KY 



A SOCIAL SY^T^ItS APPROACH TO EVALUATION RESEARCH 
' C.N. Olien, P. J. Tlchenor, G.A. Donohue * 



\ — 5 



UJ 



o 



. -h Criteria and proceduras far evaluation research' have- been najot j':ohcerit4 as 
la^^e 'scale and costly social actign programs have multiplied on t:he slate and 
rlc^tlona%levels. Inn:?vatlvp* approaches fo social problems that Involve complfcx 
organizations and processes raise an array of questions about the eviluatlonaL 
process. j\ 2 / Tests of Individual behavior modification are typical criteria for^ 
evaluation. -Eviluation might prov^<Je more exte.nslv^ structural infprmat-ion about 
. ho97 specific goiXs mav or m:y not have been aclileved which night nrovldc a broader 
base for Im^yrov^.ng future p. )gramc; 3/ 

/ 3cclc\l systea^ approach tp evaluation appears 'fruitful, in vIrw of the\£^t 
that social act\on pi:n[/rams recur in tha cclaext of a particular social system\^ 
often in an atmof^plvsre soCiil conflict and renslpn. ^/ Such an' approach^vlflws 
educational pro.^rams as, outcom^.s of interaction among separate, but interdepend 
' cocf al subsy^teiis., such 'as s^yjce , channel and ^audience 5/ and may be Contrast^ 
^ vith a more ;f requent tendency co concentrate on thie flnfil outcome of ©ci 

delij^ery based on whether giVen behavioral objectives, are achieved. 6/ y<^v example, 
tilm- evaluatlLons are* <^enerally basejl^on measured respons'es of the clientel 
viex-Tlng it and concentrates primarily on the' point of contact and on cllente] 
response alone. 7, 8, 9/ s« v 

A social system as used here refers to a pattern of Interacting and inter- 
dependent roles, and may be vl^we3 as containing a variety of, distinct but function- 
ally interdependent subsystems. 10/ Relevant subsystems for educational cowmunlca- 
^'tion would Include* a source subsystem of disciplinary specialists, .a program ^ 
7 production and delivery subsystem, and an audience subsystem. 11/ Specification of * 
^ subsystems dep^ncls upon the particular program under evaluation. \> - 
i, , / ^* . *^ 

; • Identification of interdependent segments, or subsystems, is c^n^ral to an 

[ analysis of information control processes which in turn may determine the-putcome 
. -of an educational nrogram. 12/ 

Such information controls" are exerted at different organizational levels. 
In a study of schools in Hagerstown,> school 'administrators seemed generally more 
favoirablS .toward instructional television than teachers did, "and teachers in lower 
grades were more favprablp than those in higher grades. 13/ Greater administrator-^^ 
teacher similarity at lower grades may result from the greater direct control by 
administrators at the eleitentary level. . ^ 

' in a lUnnesota study concerning science communication through printed media, 
^reporter's tended t6 produce more understandable and accurate .messages when they 
operated in an environment of rather rigid controls. 14A These control factors 
included specific editor assignment, dependence upon prlpr printed materials such 
as. presa^ releas^^ 6r journal articles, and contact with sources having administrative 
responsibilities and experience. * • ' * 

Importance of ^ the organizational structure in determining content of messages 
is further illuminated in Cantor's study of- 24 producers unji script writers who 
Specializes In chii.dren's^televioion programs.* 1^ While ail parts of the syiTtem ^ , 
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may- affipcf content, she poirfts out, >the niodt injportant Influence is from those 
segments in, direct interaction v;ith the vn?rtf«., Sh^ frequently found script 
XTritei;s treating, violence not in terms of their ^av/n feelings about it, buJr'rather 
In line yith station dr, sfetwork policy. Her finding is similar to'that of Donohevri' " • 
-•lo reported a-close corr^aspndence between attitudes of Kentucky newspaper publishers 
^l-Ti the way in uhich their* newspapers covered the Medicare issue. 16, 17/ In the, 
•ionohew study, ptAl'lsTiers ' attitude^ and content of their- papers were most anti- 
Ixaiicare in small rural towns where one 'might have (in the early 1960*3 when the 
stady'uas done) expected stfong opposition from the local medical profession. 



Evaluation and social , conflict ' 

-■—3 — ~ 

Tension, arising f rom^comjietition, deviance or lother* forms of 'social conflict 
is a characteristic^condition of soci^il systems. 18'»19. / ^o'^^t an^ tytre of tension 
is related to the nature of the social structure. For.examp]^e, , the more the 
horizijhtal segmentation amont> source specialis.ts prpduding a program, the greater 
the likelihood of differences among specialists about form and' content of messages. 
Similarly; conflict raa^ bfe related to vertical plur^lism^ Tlife .greater the number^ ^ 
of administrative levels. in program develoomeht, the ^teater^the opportunities for 
differences to occur. / , • ' 

Alfeg with identification of subsysterps, systems analysis involves * 
identification of jqoTjfllcts and ^tension and "possible- consequences > for program • 
qutconlP.^ Xrnsion by itself is noti necessarily, either negative or positive in its 
consequerf^es,. 20/ Conf\Lict and tension may be f unctionar for system maintenance, 
for.goar ajttainment, or for adaptive adjustment. In a contmunity , 'the conflict 
atmosphere surrounding an issue may leaci tq sharply focused media attention on a 
topic and an attraction of attention among,, audience segments who might otherr^ise 
ignore su(:h/ a topic'. The outcome may be widespread familiarity and a more equal 
d is t rib tit ion of understanding aaross educational levels than would *oth<^rT7lse occur. 
Several analyses of data have demonstrated that* corif lict can ^ serve to reduce 
knowledge gaps within a community. 211 , . ' 

Conflicts may\Kave a variety of consequences «.f or' program outcomes. Film 
^roduders" may defer to^ sources on wliat is; said about a topic, but argue vehemently 
over what topics to cover. Also, film-makers may have* quite different views than , 
source^ have about what cultural symbols to use and uh^t Social values to highlight ^ 
and reinforce. > Chemist^ and cin(?mato^aphers way in pxiinctple defer to each other' s"* 
professional expertise,- but there may be a large gray are^K^in message production 
that- doesn't fall neatly within one discipline or the ojimer. Chemists are concerned 
about how their profession is portrayed and cinemato^.rapher*s havet^tandards for 
dramatic and visual quality. The two concerns might "ccme^in conflict, as when the , 
cineipatrographer wishes to dramatize a scene involving 'a^arcotics addict and ^ 
medical expert in a way that the chemist f inds ^of f enslve^ to- the Research profesfltlons, 
I'Jhile such conflicts in television production may terid,?|y be resolved in favor of 
organizational policy, Mf^s Cantor found, their existence may also be functional for 
creative production. 22», 23/ ^ * ' * . ' 

PThether a conflict is or Is not functional 'Is itself ^n important- aspect of 
systems analysis in prograp evaluation. For example, extension agents and a sample 
of community editors were asked identical questions about adult education programs 
m c6mmunity development. Both groups agreed that some sort of community, planning 

. •' ■ ■ 3 . . . • ■ 
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^ was Ttieeded, On specifics, however, .there were sharp differences. While agents 

v7ero oriented toward the University educational mission, editors viewed Rural Areas 
^Development and Poverty Programs^ as largely federal efforts for pouring 'funds into 
financially-depressed to\ms and regions. Editors and agents had -the most ' ^ 
discfrepant views on topics that were most relevant to their relationships 24/ In 
this case, fundamental communication blockage. existed between, two key communication 
• roles in the system. 25/ • . • 

t • " . 

Audience assessment JSip 

'Ability f> assess audience response by a given subsystem — sUch as a source or 
channel ^roup--is another aspect of system analysis. Research evidence to .date 
suggests th^t professional communicators are '?iot necessarily more effective than 
subject matter source specialists in. estimating audience response^ to messages. 

Several studies have. pointed to the marked ability of sourc*? roles to 
estimate audience response rith relatively high degrees of accuracy. In Tannenbaum's 
' ^ studies with small purposive samples, scientists (as well as science OTiters) were 
more accurate in estimating science reader Interest in news stories than uere 
editors. 26/ Also, in' a Minnesota study of adolfesceht .reaction to two ETV films, 
meteorologists were more accurate ^ predicting audience reaction to a. film on 
v/eather and, meteorology than were professional photography experts. 9JJ (TABLE 1) 

Identification of audience assessment errors through/systems evaluation 
migh^ well lead to adjustments that improve ability^of suWystems to predict audieiace 
response. Given an opportunity to study and know a parti^ul^r community, channel 
communicators may be quite accurate In festimiting certaiji aspects of popular reaction 
,to a mass media message. . Cleary and Beal gave extensile information abou^ a village- 
and^a local water controversy to' 21 television producers. Reactions of local TV 
vifewers were then compared with producer estimates^ xThich were highly accurkte in 
predictions of audience size, audience evaluation^ ^^d amount of audience talking 
about the^^show. 28, 29, 30, 31/ , * / 

' • * ' 

System Legitimation • ' . 

. . - / • ^ / . I 

One stage, or key event in systems analysis'* is legitimation. 32/ Sesame ' 

' . Street, for example, was socially legitimized by''a broad spectrum^ of educational 

Thstitutions when it was first introduced .3^/ A six-part series on. tfcytrition developed 

through the Cooperative Extension Service ,netwrk for 9 to 12-year-olds vas-, widely " ^ 

praised, and thereby legitimized, by groi^fis of extensioji educators and nutritionists 

around the .nation before it was sent out/ to TV stations and schools for showing, to 

' youth groups. V^j This le<>l timet ion appears to have been instrumental in gaining 

' vridespread use of the seriels by botfi stations and classroom teachers. 

* / ' ' 

- / ' 

'A phase which will ordinarily follov; legitimation is commitment and allocation 
of orgariizational and media resources to production ahd delivery of the program.' 
Purposive messages do not move by themselves, but ar^ exchanged in a system ihpro- 
portion'to ther social enersy- generated by the various groups. The series on nutrition 
cited above required the mobilization of a wide network of agehcies, including the 
U.S. 'Department of Agriculture, state unlversitiies, state education departments, 
television stations, countv extension staffs.,. .and Elementary school districts. Promo- 
tion was both horizontal and vertical, and each form of diffusiop depended upon a 
nrior commitment at some agency level to devote a certain amount of energy to the 
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. ApnllcatlonQ of a systems evaluation mbdel 

A recent application of the systems evaluation model Is ^concerned with ai? 
' educational televlsloh^series on putrition 'for 9 to. i2-year-olds, tertned llulllgan 
Stet^;,"' In this case the design. dealt (a) the institutional context in which 

^ the prbgyrams were printed, (b) .ejtoectations of clientele performance amg^ig groups 
"^""^l^^' agehcies developing and ^delivering the series, and (c) the extent to which the 
V ' ta^eutele^group attained the formally stated objectives. 

This series of six half-liour color TV films .represents a major broadcast 
media educational -effort , tied in \d'th the Expanded Food and ^]utrition Program of 
the U.S. Department of Agricultural^ Federal Extension Service, in cooperation with 
t;he various State Extension Services. The Mulligan Stew series was prodjic^d by a . 
group of film and nutrition* specialists in p.he USDA, in 'cooperation xjith program ' 
and nutrition experts at several state universities. 

Formal objectives of the Mulligan Stew series included, princio^^^, assistance 
to young peop'le in acquiring and* applying nutritional principles for neaWtiful, 
productive individual development. Wiile th^ sSti-Ap was developed so that it could 
conceivably ''stand alone^ on prime time television,^ the stress was oh integi^ation 
either with classroom nutrition instruction or with food ahd Iieaith projects in 
such organizations as 4-R clubs. Supporting educational materials included a comic 
book vers.ion^of the 'films, a guidebook of group exercises for teachers or leaders, 
and even song books ^nd records for original tunes and jingles produced in the films. 

Casting and scripting of the' *';ulligan Stew^' series x^as based to a consider- 
able extent on use o£ t^ien-current dramatic and cinematip techniqiies for attracting 
younf? people. A number of ^Sesame Street » characters and techniques were employed.. » 
Briefly, the series was based .on a neighborhood group of six youngsters, between 
9 and -1,2 themselves, cast as the '*?lulligan Stew Gang*' in a modified adventur/^-com^dy 
i format. In each film, the 'gang*' solved^ a ''nutritional problem'* of certain • 
^ individuals or groups by applying their knot^ledge of nuTfitional principles. One 
case w^s an' oveTOelght race car driver w^o. lost hisMdriving job until the Mulligan 
Stew gang figured out his di4t probi^. [ Tliey reshaapd his wife's cooklog and his 
exercise routine sp that he slimmed down*^ wefTt>liacJ$/to driving tfie race car, and 
took first place oast the checkered flag.* , <- 

. ^ * V : 

SymBolic renresentatiohs and , character roles^in the series are clearly 
, calculated to appeal^to dominant value s^of American society.. The films^ emphasize, 
" .repeatedly, such virtues ^as good health7 physical fitness, clean living, school , ^ 
' achievement, athletic participation, respect for institutions and support of others 
in time of. need. Various episodes make liberal use of both positive and negative 
sopial models. Through rapid-fire sequencing of scenes, the films repeat, at 
numerous intervals, certain slogans intended 'to instill the idea of a .balanced diet^ 
the importance andf function^^of different food ^utrients, and related nutritional 
* principles. * " * " 

* ' While a great deal oFf *!)rcrfessional expertise* wa^ mobilized for production of 

the series i the planning *^d production was not an'feotirely harmonious* process. 
There were, for^^xample, some sharp divisions of opinions "between some groups- of *. 
nutritionists, extension program administrators, and production nersdnnej. over both 
. content and mode of present-tftion. Analytically, however, the»e divisions seemed to 
■^attnact pro'fessipnal attention to the project ^and ^stimulate interest in *'its „ , . " ^ 
completion. * ^ • • 

' . ^ ■ . -5 • " 
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^ The Mjuliigan-Stcw series was*released for national use In an, attno4pfiere »of 
,o{)tiinisin and high promise. The series was. made available In each Qt7ate*thr9ush 
state university extension ^services who In turn contacted county es^tenslon offices 
and school dlstrllfts, with promotional material -describing the series and the 
supporting project material* 

" 9 

$ 

u In rdnnesota^a total of 2,546 -schdols requested supporting materials for 
more than 100» 000 'elemenTary schooX students /Wstl^^^ The 
series was' scheduled on 6 commercial television channels and 4 educational channels 
for a^slX'-week period 'In October and November of 1973. Supporting materials v/ere 
closely tied to the series, so. tha.t after each, film, the teachers could select from 
a variety of specific exerplses linked to the themes of that film* 



among? 



Evaluation of this series, In the Tllnnesota study, was based upon measures 

1. A total of a?* nutritional research specialists In 3 mldwestern 
Universities, all of vrhom were familiar with the series and many of 
' ' whom. had taken part in planning of educational programs In their 
states which woXild utilize the films. 

1. Eight 4-H Information specialists, In the same 3 states, who 
* were responsible for coordinating and distributing educational 
broadcast materials- relfi^vant ^o 4-H programs. 

3. Eleven television /station representatives In the 10 stations In 
•lllnnesota and adjoining-states that broadcast the series Into 
* lllnnesota. These station representatives had originally- received the • 
f llms.j and 'had i:ooperated trtth one' of th^ 4-H Information specialists 
»in scheduiln^the serlea ^s a public service activity. 

' 4r A total of i7 classroom teachers lt| schools In v^hlch student 
reaction to the series was studied In detail-. * 

5, More than 1,000 4tfi, 5th, arid' 6th grade children In eight 
different schools In which the series was supported in varying. - 
7 degrees* These schools included a before-after control school in 
an area of ikscbnsin x/here the series was not available, a before- 
after control where the series was on^TV but no supporting m»terlals 
were used, and- three schools,where the series was on TV and'^elther - , 
- moderate or "heavy use of supporting matej^ials were, used along with- . 
in-class videotapes of the ;5erie's. *One< school served as an after- ^ 
only control. * . ^ . 

A few general hypotheses guided this evaluation. One was that among the^ 
various non-audier^ce subsystem.ei those most closely associated with the original 
spbnsoring agency I'ould tend to have the lilghest expectations of audience response, 
compared with subsystems more clOBcAy associated with the client audience. This 
• meant tliat nutrition specialists and 4-H info^rmatlon specialists would be expected 
to have higher levels of , expectations than would ^,the TV or teacher groups. 

A second ^lli'ding' Kypo.thesls was that the more closely associated .a^ prior 
subsystem is with the audience subsystem, the more accurate the perceptions of 
audience reaction and behavior 'will be. According to this reasoning, the teachers 
"would tend to haye^'tjie most accurate perceptions 'oiE. reaction to the films. 
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^Finally, audience reaction was expected to relate cfirectly to control factors 
In the' final delivery situation. Specifically, the more extensive the control bver 
the message enyironraent in which the purposive message is presented, the higher the 
level of achievement of behavioral objectives within that audience. Op'erfitionally , 
this meant thai student learning and behavioral change (more regular eating of 
balanced diets) would be highest in. schools which used the m6st supoprtlng materials 
and re-rau the films on videotape l<i the school. , ' * * 

« «' 

^ Flndlnt^s frow the Mulligan Stew evaluation' 

'\ .' *" ' / ' ' ' ■ - ' 

^ ' The t^lndings were 'partially consistent V7lth the firat hypothesis, in the> 
S^hse that 4-n information Specialists made the most optimistic estimates^ of audience 
reaction. Thls^flnding is entirely in line with the fact that the -information 
specialists \mre more closely identified with the USDA film program^unlt that 
produced the series than was any other subsystem in (the study. Contrary to 
expectations, however, the nutritionists tended to. give far more conservative 
estimates of student reaction than information specialists did, 'perhaps , 
reflecting the fact that there were differing professional vi^ws on what nutritional 
content sl\puld be contained in such a series. Among the 37' nutritionists, six . 
identified content whlch^ in their judgment, was either partially or entirely 
inaccurati, and three tpok strong exception to what the considered to be improper 
po'rtrayal of the teacher role in the .films. 

The second hypothesis was supported, in the sense that teachers and television 
• station personnel tended to mSke more -accurate estimates than^^U information 
SFeclalists did. On the other hand, the nutlfitionists r presumably farther removed** 
from the young population in this coraraunication /chain, are nevertheless close to 
st-iident estimates.. , ' ' 

By averaging the absolute discrepancies between ratings across the 7 Items, ^ 
it is possible to make an overall comparison of accuracy estimating student 
reactions, as in the bottom line of table 2. Overall, the television station 

'personnel made the most accurate estlniates, followed by the nutritionists, teachers, 
and information specialists, in that order. These findings may be viewed in light" 
of previous findings, such as those of Tannenbaum, who found that scientists in 
some cases may predict reader- values about*. science news more accurately than editors 
do. 36/ Similarly, in an earlier ilinnesota study, a group of photography ' 
professionals were -less accurate than source specialists In estimating audience 

-evaldatldtis of Interest. 37/ In the 'lulligan Stew study, a sharp line may be ^ 
dra\m .between media perapnnel an*' the information specialists who, in this case, 

,make quite different 'estiinates, which may well reflect their differing location in 
the entire system. ' . 

THe third hypothesis, that learning and behavioral change would be related to 
information controls at the point of delivery, received jstrong support'.' In both 
•before-after control schools* — the on'e where the films' were not on TV and the one 
where the f llmg were available b*ut not supported in the 'classroom — there was no 
observed change in either measured nutritional knovrledge (based on a 17 item 
nutritional knowledge scale) or in reported eating of balanced diets. Sharp changes 
occurred in the three schools where there was moderate or heavy In-school support 
of the films. ^Two-V7ay analysis of variance showed a highly significant difference 
among schools (p < .001) but no difference by grade. There was, however, a sharp 



difference across classrooms/ according to teacher Input; The nur^ber of nutrition- 
al exercises -used In the film la a rather powerful predictor of knowledge change 
across the 14 classrooms in the three schools with moderate td heavy, use of 
supporting materials* T^e rani: correlation for these 14 rooms Is ,82 (p<; < -Ol)..^ 

>• ^ 

Importance of such direct control may be highflghted by the further finding 
that student exp*resslon of Interest In the series was not highest In the schools that 
»had the highest viewing or the highest c^ain scores* Furthermore, ^dthln schools, 
gain scores were generally uni^elated^ to evalua^tlons of the films or the Mulligan 
Stew actors-. ^ ' - . ' 

In general, then, the Mulligan Stew evaluatlonal study provides strong su|$port 
for an evaluatlonal approach based on a model of information and knowledge, control. ■ 

.To the extent that knowledge is produced', packaged, and delivered In the Inatltutlon- 
al educ^itlonal setting with full deployment of " institutional resources, there is a 
greater likelihood of learning and retention of content- and of change in' related 
behavior. On the other hand, extensive avHilabll3,ty-of such ' content 'On the mas^s^ 

'madia channels alone does 'not appi»^r to meet the conditions necessary for the 
Intended result.. ► ^- ^ . ^ 

Responses of the non-audlenc'e! groups may be seen in' a systems^ perspective in 
terms of^ their predictive value, as contrasted wlth^ their controlling Influence on 
final outcomes. Givep the key Ipcatlon of information specialists in the ar.ganlz^t- 
lonal netu^orlc distributing'the series, their estimates most likely indicate an early, 
and deep professional commitment to. the ^entire project. Fr6m an analytical 
perspective, these responses n\^y predict more about the use of the prograp In the* 
total system than about ultimaife 'audience respolcise. • . ' 

?tost accurate from a predictive point of view are the television/^'statioh 
representatives' estimates, at least for young 'people who. in fact. see the series. 
But actual level of viewing of an avowedly purpdslve broadcast message" of' this type 
requires organizational control at the audience subsystem^ level as a necerfsarjr, 
condition* This is not* to say that school control is the only means' of meeting 
this conditio \: 4-11 club organizations' or other parent-youth groups might well be 
able to provide functionally equivalent organized attention. . 

Summary ^ - . ' ^ . , 

f • " ' - . * ' ' • • - . 

Aa Ipformatlon-cintrol systems model for evaluation of adult education 
progr.ams is offered and illustrated^ The' model .is based upon identifying principal 
subsystems, such -as source,, channel and audience, :;hlch are involved in initiation, ' 
production, delivery and-receptlon of* educational messages. These subsystems are 
seen as separate but interdependent, having the power singly or Jointly to make 
crucial. decisions about generation, dissemination or withholding of Infonnation^^ 

I ^ ^ 

The extent to which these subsystems may in fact ^xert ,such controls has' 
been demonstrated in a variety of research studies, many in the mass communication 
area. Control over^ content ,^ fpr example, may relate to media leadership ties* with 
the community leadership, and to links betv/een media personnel and source systems. 
Similarly,. pource and channel subsystems have differing ablll.tles to predict levels 
and" type of potential audience response to messages; in seyer^l studies, "scientist 



or ot'her specialist sources could make^ such predictions uith l^els of accuracy 
•either similar to or ffreatCr than pred'itrtions by some mass media groups. 

, . ^ ^ 5^ 

Specific application of the model is illustrated in evaluation of the 
•'Uulligan Stew'' television series which-, although directed at a young audience, 
has implications for evaluation of a1;iy program of adult education which involves-/' 
extensive use of the^mass conmunication media. .The study, included before-afte^V 
student responses in eight schools and estimates of such responses iErom nutrition • 
specialists, information specialists, television station personnel, and teachefk. ^ 

Among estimates of audience evaluation, the most opt'imistic were mad^ by . 
information specialists who were clod^ely tied to the sponsoring agency. The most 
ac^cu^ate estimates of audlehce reaction x^^re made by television station personnel. 
Close in accuracy to the station personnel were nutrition specialists, providing 
further support for the generalization that professional research .groups may be 
highly accurate in predicting .audience response to popularized versions of content 
in their specialized fields. 

In terms of final outcome, the Muliigan Stew results generally and strongly 
support the information control systems model, and testify to 'its utility in 
evaluational research. The factor most closely and immediately related to knowledge 
gain is organized input at the. school and classroom level; broadcasting the series 
onjpri^e time television (Saturday morning) :hours for the target audience without 
iri^schbol support produced no measurable change in knowledge change or related 
behavior. • - , . " 
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